Parallel Programming with Control

Parallel programming involves nding the potential parallelism in an application and mapping it to the architecture at hand. Since a typical application has more potential parallelism than any single architecture can exploit eeectively, programmers usually limit their focus to the parallelism that the available control constructs express easily and that the given architecture exploits ee-ciently. This approach produces programs that exhibit much less parallelism than exists in the application, and whose performance depends critically on the underlying hardware and software. We argue for an alternative approach based on control abstraction. Control abstraction is the process by which programmers deene new control constructs, specifying constraints on statement ordering separately from an implementation of that ordering. With control abstraction programmers can deene and use a rich variety of control constructs to represent an algorithm's potential parallelism. Since control abstraction separates the deenition of a construct from its implementation, a construct may have several diierent implementations, each exploiting a diierent subset of the parallelism admitted by the construct. By selecting an implementation for each control construct using annotations, a programmer can vary the parallelism in a program to best exploit the underlying hardware without otherwise changing the source code. This approach produces programs that exhibit most of the potential parallelism in an algorithm, and whose performance can be tuned simply by choosing among the various implementations for the control constructs in use. We use several example applications to illustrate the use of control abstraction in parallel programming and performance tuning, and describe our implementation of a prototype programming language based on these ideas on the BBN Butterry.